Setting up a Tumblr-to-Misskey crossposter

Recently I reactivated my Misskey instance (yuri.doll.gl) and was struck by that most weary of sensations all selfhosters experience at some point: "by god, how do I keep up with all of these?" And so, to make my posting efforts marginally less tedious, I decided to set up a crossposter between my Tumblr (which is my favored social media) and my Misskey.

The first thought I had was setting up something to copy my posts from Misskey to Tumblr. This ended up being a bit of a bust, as I would be relying on Cloudflare Workers to handle scheduled job running (I'm far too lazy to spin up a persistent process on my VPS!) Cloudflare Workers, unfortunately, does not provide access to most default Node.js modules, such as those required for the OAuth authentication system used by Tumblr's API. So I turned my attention to the reverse scenario- Tumblr to Misskey.

Here I had two choices. Tumblr's API allows for reading posts without full OAuth authorization, but I fancied myself clever and decided to use a somewhat novel approach in the experimental Discord webhook feature. This feature, once enabled, allows you to send your recent posts & reblogs to a Discord server of your choice, like so:

Quite neat! Of course, now the problem changes to how a Cloudflare Worker can read these messages and convert them to a Misskey-postable format. Thankfully, Discord's API is much simpler to authenticate with than Tumblr- all one has to do is create a bot, add it to your server with message read permissions, and then use the provided bot token to fetch messages from the designated channel.

Misskey's API is likewise very simple- quite similarly, all you have to do is create an API key from your settings page and use it to authenticate against your instance's endpoint.

Now, all that remains is to put the pieces together. First, you create a new Cloudflare Worker, alongside a KV storage namespace (I'll explain why shortly!) and a Cron job schedule (I set mine to once per minute).

The first bit of code is the definition of constants like the various API keys and endpoints we'll make use of.

const DISCORD_BOT_TOKEN = 'your-bot-token';
const CHANNEL_ID = 'your-channel-id'; 
const MISSKEY_API_URL = 'https://your-instance/api/notes/create';
const MISSKEY_API_KEY = 'your-api-key';

Then we add handlers to activate when the scheduler calls:

addEventListener('scheduled', event => {
  event.waitUntil(handleScheduledEvent());
});

async function handleScheduledEvent() {
  const messages = await fetchDiscordMessages();
  await checkForNewPosts(messages);
}

Here's the bit that fetches messages from your specified channel via the bot:

async function fetchDiscordMessages() {
  const response = await fetch(`https://discord.com/api/v9/channels/${CHANNEL_ID}/messages`, {
    method: 'GET',
    headers: {
      'Authorization': `Bot ${DISCORD_BOT_TOKEN}`,
      'Content-Type': 'application/json',
    },
  });

  if (!response.ok) {
    console.error('Failed to fetch messages from Discord');
    return [];
  }

  const messages = await response.json();
  console.log(messages);
  return messages;
}

Now that we have a list of recently posted messages (50, to be exact- if your designated channel gets more than this amount of messages per minute, I recommend setting up a dedicated empty channel for your crossposter), we can filter it by sender (the Tumblr webhook). Because the posts/reblogs are displayed in the form of embeds, we simply have to access the content via message.embeds[0], like so:

async function checkForNewPosts(messages) {
  const recentMessageIds = await getRecentMessageIds();
  for (const message of messages) {
    if (message.author.bot && message.author.username === 'Tumblr notification about your-blog-title' && !recentMessageIds.includes(message.id)) {
      let postContent = (message.embeds[0].description || '') + "\n";
      if (message.embeds[0].fields) {
        postContent += (("tags: #" + message.embeds[0].fields[0].value.replaceAll("\ ", "\_").replaceAll("\,\_", "\ \#") + "\n") || '');
      }
      postContent += (message.embeds[0].url || '');
      await postToMisskey(postContent);
      await updateRecentMessageIds(message.id);
    }
  }
}

You might have noticed the presence of a certain getRecentMessageIds() and updateRecentMessageIds(). This is where the KV namespace we made earlier comes in! Without it, we have no way of checking that a message was already posted to Misskey, leading to unwanted duplicates. To avoid this, we store the 50 most recent Tumblr messages' ids in the KV store and check against it every time we want to post something.

async function getRecentMessageIds() {
  const recentMessages = await KV_NAMESPACE.get('recentMessages');
  return recentMessages ? JSON.parse(recentMessages) : [];
}

async function updateRecentMessageIds(newMessageId) {
  let recentMessages = await getRecentMessageIds();
  recentMessages.unshift(newMessageId);
  if (recentMessages.length > 50) {
    recentMessages.pop();
  }
  await KV_NAMESPACE.put('recentMessages', JSON.stringify(recentMessages));
}

The KV_NAMESPACE here refers to the namespace we created earlier, assignable to this Worker via its settings.

Now that we've gotten getting the messages out of the way, all we have to do is turn it into a Misskey payload. This is pretty simple; since we already have the post content, just stick it in a POST request!

async function postToMisskey(content) {
  const misskeyPayload = {
    visibility: 'public',
    text: content,
    localOnly: false,
    noExtractMentions: false,
    noExtractHashtags: false,
    noExtractEmojis: false,
  };

  const response = await fetch(MISSKEY_API_URL, {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      'Authorization': `Bearer ${MISSKEY_API_KEY}`,
    },
    body: JSON.stringify(misskeyPayload),
  });
  console.log(response);

  if (response.ok) {
    console.log('Successfully posted to Misskey!');
  } else {
    console.error('Failed to post to Misskey');
  }
}

You can now check whether your crossposter is working properly using the "Trigger scheduled event" button in the Schedule tab, located on the right-side panel. If the console shows the success message ("Successfully posted to Misskey!") and your Misskey instance correctly displays the posted note, congratulations- you no longer have to worry about keeping your Misskey in sync with your Tumblr!

The full code is shown below for reference. Happy crossposting!

const DISCORD_BOT_TOKEN = 'your-bot-token';
const CHANNEL_ID = 'your-channel-id'; 
const MISSKEY_API_URL = 'https://your-instance/api/notes/create';
const MISSKEY_API_KEY = 'your-api-key';

addEventListener('scheduled', event => {
  event.waitUntil(handleScheduledEvent());
});

async function handleScheduledEvent() {
  const messages = await fetchDiscordMessages();
  await checkForNewPosts(messages);
}

async function fetchDiscordMessages() {
  const response = await fetch(`https://discord.com/api/v9/channels/${CHANNEL_ID}/messages`, {
    method: 'GET',
    headers: {
      'Authorization': `Bot ${DISCORD_BOT_TOKEN}`,
      'Content-Type': 'application/json',
    },
  });

  if (!response.ok) {
    console.error('Failed to fetch messages from Discord');
    return [];
  }

  const messages = await response.json();
  console.log(messages);
  return messages;
}

async function checkForNewPosts(messages) {
  const recentMessageIds = await getRecentMessageIds();
  for (const message of messages) {
    if (message.author.bot && message.author.username === 'Tumblr notification about your-blog-title' && !recentMessageIds.includes(message.id)) {
      let postContent = (message.embeds[0].description || '') + "\n";
      if (message.embeds[0].fields) {
        postContent += (("tags: #" + message.embeds[0].fields[0].value.replaceAll("\ ", "\_").replaceAll("\,\_", "\ \#") + "\n") || '');
      }
      postContent += (message.embeds[0].url || '');
      await postToMisskey(postContent);
      await updateRecentMessageIds(message.id);
    }
  }
}

async function getRecentMessageIds() {
  const recentMessages = await KV_NAMESPACE.get('recentMessages');
  return recentMessages ? JSON.parse(recentMessages) : [];
}

async function updateRecentMessageIds(newMessageId) {
  let recentMessages = await getRecentMessageIds();
  recentMessages.unshift(newMessageId);
  if (recentMessages.length > 50) {
    recentMessages.pop();
  }
  await KV_NAMESPACE.put('recentMessages', JSON.stringify(recentMessages));
}

async function postToMisskey(content) {
  const misskeyPayload = {
    visibility: 'public',
    text: content,
    localOnly: false,
    noExtractMentions: false,
    noExtractHashtags: false,
    noExtractEmojis: false,
  };

  const response = await fetch(MISSKEY_API_URL, {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      'Authorization': `Bearer ${MISSKEY_API_KEY}`,
    },
    body: JSON.stringify(misskeyPayload),
  });
  console.log(response);

  if (response.ok) {
    console.log('Successfully posted to Misskey!');
  } else {
    console.error('Failed to post to Misskey');
  }
}