Steemex v0.12 released: now with improved blockchain event pipeline [open source Elixir library for Steem]

in #steemdev7 years ago


I've been working to improve the developer experience for Steemex and this release does exactly that. I had to introduce some breaking changes but it was worth it. Stage.Blocks, Stage.RawOps and Statge.MungedOps GenStage producers now emit events with the following shape : %Steem.Event{data: ..., metadata: ...}` where data is a new Steem blockchain operation struct (or a map) and metadata includes timestamp, block height, source blockchain, type and if the operation data was munged.

All event producers follow GenStage spec which allows you to easily patch into the transformation pipeline at any step: block producer, ops producer or munged ops producer.

Repos and docs

Changelog

  • breaking: event shape for GenStage pipeline changed from tuple to %Steemex.Event{} struct
  • breaking: all API responses are now structs or atom keyed maps
  • all stages now emit %Steemex.Event{data: ..., metadata: ...}
  • added Steemex.Block struct
  • added Steemex.Event struct
  • Steemex.Stages.Ops.ProducerConsumer renamed to Steemex.Stages.RawOps
  • Steemex.Stages.StructuredOps.ProducerConsumer renamed to Steemex.Stages.MungedOps
  • timestamp strings are now parsed into NaiveDateTime
  • example consumer was updated to match a new shape of events
  • multiple various code improvements

A GenStage consumer example

defmodule Steemex.Stage.ExampleConsumer do
  use GenStage
  alias Steemex.MungedOps
  require Logger

  def start_link(args, options \\ []) do
    GenStage.start_link(__MODULE__, args, options)
  end

  def init(state) do
    Logger.info("Example consumer is initializing...")
    {:consumer, state, subscribe_to: state[:subscribe_to]}
  end

  def handle_events(events, _from, state) do
    for op <- events do
      process_event(op)
    end
    {:noreply, [], state}
  end

  def process_event(%{data: %MungedOps.Reblog{} = data, metadata: %{height: h, timestamp: t} = metadata}) do
      Logger.info """
      New reblog:
      #{inspect data}
      with metadata
      #{inspect metadata}
      """
  end

  def process_event(%{data: data, metadata: %{block_height: h, timestamp: t} = metadata}) do
      Logger.info """
      New operation:
      #{inspect data}
      with metadata
      #{inspect metadata}
      """
  end

end

Roadmap

  • Add latest blockchain ops
  • Add more structured ops and transformations
  • Add transaction broadcast

About Elixir

Elixir is a functional programming language with superior concurrency primitives, distributed computation and fault tolerance capabilities based on Erlang/OTP.
Personally, I am really happy with my bet on Elixir to build cryptotokens related apps and strongly recommend every developer to try it.

Learning resources:

(∩^o^)⊃━☆゜.*

Sort:  

Steemit is getting better and better. That's the only thing I understood from this post :)

Jeeeee. At least your hinest. Yes it is so invest everything into steempower because I dont see it going below 2.00 ever again.

Domi

Steem, but yes :D

Appreciate your work and helping grow our community.

(∩^o^)⊃━☆゜.*

def process (%{data:awesomoness}

  """

end

Guys Elixir is programing language from what i understood. The writter is talking about how it is easy to use and is helpful for creating token apps. right ?
@ontofractal?

This one is interesting and have something great information
Thank You for sharing
And serving steemit community

Wow, thats a good news i think

"I've been working to improve the developer experience for Steemex and this release does exactly that."
How long did that working last to get the release through?
Might look an eye on the mobiel app too ? ;)

This is a library for developers, not a user facing app though :)

great info! thanks for sharing

I apologize for my blockchain-programming ignorance, my 2 basic questions are:

  1. will censorship & security issues be addressed?
  2. will the endusers like me see filtering & search features alleviate the congested resteemit mess that clog up flow to original posts on profile pages?

Like I said I do truly apologize if the above is unrelated, but perhaps your programming buddies are looking into these issues and you could discuss it as a seamless fit or transition.

Thanks for sharing and thank you for being here!

Thanks to everyone who work to improve the Steemit experience.