What is the Level of Human Engagement on Steemit? 15%?

Engagement on steemit is a key component to its success.  Someone writes a posts and others comment and engage.  Discussion are created.  But how well are people really engaging on posts? 

 Repository

https://github.com/steemit/condenser 

Aim of Analysis

The aim of this analysis is to establish a trend for the number of comments per post over time and also to get an indication on what level of comments are human comments and human engagement. ( It has been requested a number of time to estimate the bot activity)

The Findings

The pie chart below is a simple representation of the number of comments left on all posts since January 2017.  Originally I plotted a histogram to plot the distribution of comments but as 80% of posts get less than 5 comment, the chart was skewed too far to the right to be even visible.

  

Take a look at the image below.  It shows in the blue bars the average comments per post per month since the start of 2017.  The black line shows the number of posts, the red line shows the number of comments(children) and the yellow link shows the unique authors.

 

The data looks interesting and in terms of engagement rather positive really.  It looks like the average number of comments per post bottomed in November with an average of only 3 comments per post. Since then the number of comments per month has increased massively and the average comment per post was up to 3.3.  Looks like good news so far for engagement.

 

Have a look now at the GIF above.  It shows the same data, this time removing the posts with no comments, the posts with less than 2 comments, then less than 3 and then less than 4 and 5 as these make up 80% of the posts.

The GIF shows that when we exclude posts that have less than 2 comments (which is 60% of the posts) the average number of comments per post increase to 8 which is rather stagnant since Nov 17.

By the time we remove 80% of the posts, the average comments per post increases to 14.5.

Overall however the trend remains very close.  There was a decrease in the number of comments per post from June 17 to Nov 17 and since then it has gained ever so slightly.  Nothing to write home about.

There are many problems with comments or lack of good comments left on post, but there is also a problem of bots.  Bots leave comments for many reasons, we have welcome bots and bidbots and flagging bots all leaving comments.

Adjusting for Bots

Lets take a zoom in to March and April this year and we are going to filter the data so that we are only looking at posts that have at least 1 comment.

  

How different would this data be if we removed the comments from the bots?  To get an indication of this I took a random sample of well-known bots.  

@minnowsupport @cheetah @steemcleaners @randomwhale @booster @buildawhale @welcomebot @randowhale @randowhale0 @minnowhelper @randowhale1, @drotto.

This sample represents bidbots, support projects and spam filters

We all know 12 is a very small sample, but lets take a look at the data now, excluding comments from these particular bots.

  

Can you see what happened there?  Let’s look at the same information in table format

  

Looking at all posts with 1+ comments, if we remove the comments from the bots listed we remove 73% of the posts and 85% of the comments in March.  On what is left, the average comment per post reduces from 5.73 to 0.84

Conclusion:

Looking at Steemit in general the number of comments per post has generally decreased since January 2017 and the % of posts with 1 or more comments averages at 58% and with 5+ or comments is only 20%

However as everyone is aware since 2017 there has been a substantial increase in the number of bots leaving comments.  If we remove all of the comments from the bots listed, the results are not good.

This estimating is showing that the number of posts with 1+ comments reduced by 73% if we remove comments from bots.  Therefore 73% of posts have received a comment from a bot.  The total comments has reduced by 85%.  That is massive.  That suggests 85% of comments are left by bots.  Suggesting only 15% of comments are human

Data and Query

Although I was expecting a high level of activity from bots, I was not expecting it to be that high, so below are my workings and queries.  Please have a look at this carefully and tell me if you spot anything wrong.  I would love to be proved wrong on this.

To get all the Comments data the following query was used

SELECT author, root_title, children, created, depth

FROM comments (NOLOCK)

where created >= CONVERT(DATE,'2017-01-01') 

I then used the following calcuations

No of Posts = COUNTROWS(FILTER(comments,comments[depth]=0))

No of Comments = COUNTROWS(FILTER(comments,comments[depth]>0))

average comments per post = [Comments on posts]/[No of Posts]

these calculations were further filtered to exclude posts where the children was = 0 to get the values of posts with 1+ comment.

To get data excluding bots I carried out the following query:

SELECT author, root_title, children, created, depth

FROM comments (NOLOCK)

where created >= CONVERT(DATE,'2018-03-01') 

and author not in ('minnowsupport' , 'cheetah' , 'steemcleaners', 'randomwhale', 'booster', 'buildawhale', 'welcomebot','randowhale','randowhale0','minnowhelper','randowhale1','drotto')

Calculations

Posts = CALCULATE(DISTINCTCOUNT(comments[root_title]),comments[depth]<>0)

No of Comments = COUNTROWS(FILTER(comments,comments[depth]>0))

A comment per post = [No of Comments]/[No of Posts]

I then added a filter to each of these calculations to exclude posts where the children was = 0