SteemSQL - Infrastructure upgraded for better performances

arcange (73)in #steemsql • 7 years ago (edited)

7 years ago in #steemsql by arcange (73)

$80.34

Sort:

Trending

[-]

bronevik (61) 7 years ago

Great work, @arcange!

Few advices that could help.

Read-only replicas could offload a great portion of read requests and greatly reduce locks. Master node should only write and send WALs to replicas, so, no locks there essentially. And replcas could be cheaper or even community-maintained.
Most of the queries are probably made againt last week data, so I would expand by building a separate nodes containing only a portion of data. Ideal solution - JSON handlers on fully RAM cache. For example, storing data from last week would require 7*80MB=560MB or RAM.
With main DWH you could consider Yandex Clickhouse solution. It's free and it could serve as stream write + read cluster from-the-box and do it fast.

My upvotes, @bronevik.

$0.43

4 votes

[-]

arcange (73) 7 years ago

@bronevikm, Please apologize for my late reply

Thanks for your comment and suggestions.They are really interesting and I will analyse them in deep.

$0.00

[-]

paulag (73) 7 years ago

I knew there was a reason you have my witness vote :-)

Thank you for all of this, I am amazed at some of the stats you provided

$0.10

3 votes

[-]

arcange (73) 7 years ago

I was also surprised when checking the stats. Would I have expected such numbers, not sure i would have launchec such a project.
Anyway, it's quite challenging and I like that.
Thank for you trust and witness support!

$0.05

1 vote

[-]

eastmael (67) 7 years ago

Definitely worth of a witness vote.

$0.00

[-]

arcange (73) 7 years ago

Thanks! 🙇

$0.00

[-]

abh12345 (76) 7 years ago

Great stuff, thank you very much for this @arcange :)

The data and number of SteemSQL users is going up and up!

As an aside, I had a look online for a force WITH (NOLOCK) setup for the DBA, but cannot see anything obvious.

I do always try to remember to add it to each select.

Cheers!

$0.05

1 vote

[-]

lukestokes (75) 7 years ago

Thank you so much for providing this valuable service.

$0.03

2 votes

[-]

eastmael (67) 7 years ago

Wait, that was quick. Please confirm if I understood this announcement correctly, we can now query SteemSQL? It's now up? I was expecting it to be down for 7 more hours based on the announcement.