You are viewing a single comment's thread from:
RE: Introducing datasteem an open source block parser to store posts and user data on a mysql database.
Thank you for the contribution. It has been approved.
- What do you think of https://github.com/steemit/sbds ?
- What types of complex query are you doing on this dataset?
- How big is your database at this point?
Need help? Write a ticket on https://support.utopian.io.
Chat with us on Discord.
Thanks !
I didn't know about sbds, it looks very similar to what I'm doing I'll have a deeper look at it to see how I can contribute
Well for instance "get all the posts made by users between 30 and 50 rep, that is at least 1 hour and 6 minutes old and maximum 6 days old. With a reward between 1 and 20$. containing either the tag science technology or health but has to have the tag steemstem and show the first 450 chars as a preview and the author must not be one of those 5000 blackisted users"
For 6 days worth of data I'm hanging around 1-1.5 gb depending on the size of the text of the posts. It can take quite a while to actually query all that so I store it using the MEMORY db engine so everything gets stored in ram where I can go over the data more quickly.