If you are a developer and you are working with data from public RPC endpoints, this might be of interest for you. I've found the reason for the duplicate account operations and posts on SteemWorld and just want to inform you about the cause for the issues here.

Don't use the field 'id'

---

When you make a get_content call or any other that returns posts, the returned post-id can differ from node to node. I recommend to use the fields author and permlink for identifying posts and never rely on a (global) unique ID in the field id, because otherwise you will get in trouble for sure.

The same is true for loading data from the account history with get_account_history. The returned id might not always be the same. Even when using just one endpoint like api.steemit.com, the returned operations must not always have the same id and, as I saw in my tests, they can even differ from minute to minute. The reason for this will be explained below.

How to identify one unique account operation

---

A few months ago I changed my getAccountHistory method on SteemWorld, so that each operation contains an unique_id. The only way to achieve this is to build an own ID by combining the block, trx_id or timestamp with the operation's content. One can't rely on the transaction details like op_in_trx or trx_in_block solely, because they can differ with different node configurations.

I'm building the ID by combining the block-number with the op-type in op[0] and a 32-bit hash of the operation's data in op[1]:

---

buildUniqueId: function(blockNum, op)
{
    let i, key, hash,
    data = {},
    keys = Object.keys( op[1] ).sort();

    for (i=0; i<keys.length; i++) {
        key = keys[i];
        data[ key ] = op[1][ key ];
    }
    hash = fastHash( JSON.stringify(data) );

    return blockNum + "-" + op[0] + "-" + hash;
}

---

Sometimes there were still duplicate operations coming up in the past weeks and the reason for this was the field order in op[1], which can also differ from node to node. Therefore I'm sorting the fields by name first (as shown in the code above) and it works without any issues now.

Of course, a fast 32-bit hash may not be the right choice when storing the operations in a database, but for my purpose on SteemWorld it is good enough and does the job very well.

Keep in mind that your requests might be routed to different endpoints

---

Some RPC node maintainers do route incoming requests to other endpoints, when they make a replay or change configurations on their servers, therefore you can't say for sure that the received data always come from the same node. That is why you should never make use of the field id to get a (global) unique ID for the received data.

Hope this helps some of you.