Developing a search for similar posts. Step one: reading posts on Steemit.

Hello everyone.
Since I currently have some free time and the opportunity to work on my laptop, I'm trying to get into programming.

I've chosen the task of searching for similar posts. Or, in other words, Topic Modeling—the potential ability to structure posts by topic.

I'm trying to implement it in Python using Grok xAI.

I'd like to take it step by step to make it simpler. The first thing I need is access to the posts so I have something to analyze.

First, I need to retrieve the posts. First, a test with 10 posts:

import requests
import json

# Endpoint (stable node for Steem)
url = 'https://api.steemyy.com'  # Alternatives: 'https://api.justyy.com' or 'https://rpc.steemviz.com'

# Function to fetch discussions
def get_discussions(method='get_discussions_by_trending', tag='steem', limit=10, start_author=None, start_permlink=None):
    query = {"tag": tag, "limit": limit}
    if start_author and start_permlink:
        query.update({"start_author": start_author, "start_permlink": start_permlink})
    payload = {
        "jsonrpc": "2.0",
        "method": "condenser_api." + method,
        "params": [query],
        "id": 1
    }
    try:
        response = requests.post(url, json=payload)
        response.raise_for_status()
        return response.json().get('result', [])
    except requests.exceptions.RequestException as e:
        print(f"Error: {e}")
        return []

# Fetch posts (example: trending by tag 'crypto')
posts = get_discussions(method='get_discussions_by_trending', tag='crypto', limit=10)

# Output for verification
print(f"Received posts: {len(posts)}")
if posts:
    for i, post in enumerate(posts):
        print(f"Post {i+1}:")
        print(f"Title: {post.get('title', 'No title')}")
        print(f"Author: {post.get('author', 'Unknown')}")
        print(f"Link: https://steemit.com/@{post.get('author', '')}/{post.get('permlink', '')}")
        print(f"Text (first 200 characters): {post.get('body', '')[:200]}...\n")
else:
    print("No posts. Check the tag or endpoint.")

# Pagination (if more are needed)
if posts:
    next_posts = get_discussions(start_author=posts[-1]['author'], start_permlink=posts[-1]['permlink'])
    print(f"Next page: {len(next_posts)} posts")

Interestingly, I also made the illustration with Grok's help. I suggested he do the illustration, and then edited it.